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Abstract  -  This  paper  describes  the  use  of  Active  Shape 
Models  to  segment  the  tibia  bone  in  ultrasound  images. 
Using  CT  images  from  three  volunteers,  a  training  set 
consisting  of  60  tibia  contours,  each  represented  by  45 
labeled  points,  is  first  established.  After  aligning  the 
contours  in  the  training  set,  a  Point  Distribution  Model 
(PDM)  is  generated  that  includes  a  mean  shape  of  tibia 
and  six  modes  of  shape  variation  that  explain  98%  of  the 
total  shape  variation.  Using  the  PDM,  a  new  shape  can  be 
generated  which  is  further  scaled,  rotated  and  translated 
to  match  the  tibia  contour  in  an  ultrasound  image. 
Preliminary  results  indicate  that  Active  Shape  Models 
provide  a  useful  tool  for  segmenting  leg  bones  in 
ultrasound  images. 

Keywords  -  Active  shape  models,  segmentation,  tibia  bone, 
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I.  INTRODUCTION 

Edge  detection  and  structure  segmentation  for  ultrasound 
images  are  very  challenging  processes  due  to  the  relatively 
poor  image  quality.  Traditional  automated  algorithms  often 
fail  in  such  applications.  An  example  is  segmentation  of  tibia 
and  fibula  from  ultrasound  images  obtained  by  scanning  the 
residual  limb  of  an  amputee  for  the  purpose  of  computer- 
aided  prosthetic  socket  design  [1].  In  these  images,  the 
shading  effect  of  the  bones  causes  the  contours  of  tibia  and 
fibula  to  appear  “incomplete”  in  the  inner  portions  where  the 
two  bones  face  each  other.  In  this  paper,  we  report  the 
preliminary  results  of  using  “Active  Shape  Models”  (ASMs) 
[2]  to  segment  tibia  from  ultrasound  images.  ASMs  use  a 
deformable  template  generated  from  a  training  set  to  segment 
the  targeted  structure  from  the  image.  Statistical  parame¬ 
terization  provides  global  shape  constrains  and  allows  the 
model  to  deform  only  in  ways  implied  by  the  training  set. 
We  found  that  ATMs  are  the  suitable  choice  for  segmenting 
tibia  from  ultrasound  images. 

II.  METHOD 

Implementation  of  ASMs  consists  of  three  steps.  The  first 
step  is  to  prepare  a  training  set  that  includes  various  possible 
shapes  for  the  targeted  structure  (in  the  current  application,  it 
is  the  contour  of  tibia  cross  section).  The  second  step  is  to 
generate  a  Point  Distribution  Model  (PDM)  by  analyzing  the 


statistics  of  the  set  of  aligned  shapes.  The  third  step  is  to  use 
the  PDM  to  actually  segment  the  targeted  structure  in  an 
image.  In  the  following  sections,  we  describe  each  of  the 
three  steps  using  our  specific  examples. 

A)  Prepare  a  set  of  aligned  training  shapes 

60  CT  images  are  taken  from  both  legs  of  three  volunteers 
(two  amputees  and  a  healthy  person).  Contour  of  tibia  in  each 
image  is  detected  and  the  shape  of  tibia  is  represented  by  a  set 
of  45  labeled  landmark  points  on  the  contour.  To  facilitate  the 
alignment  of  the  shapes  in  the  training  set,  three  anchor 
points  are  first  placed  on  each  contour.  The  anchor  points  are 
generally  related  to  some  identifiable  anatomical  features  of 
the  structure.  For  example,  in  our  current  application,  the 
first  anchor  point  is  chosen  as  the  point  closest  to  fibula,  and 
the  other  two  define  the  tinning  points  of  the  bone  contour,  as 
shown  in  Fig.  1. 

Next,  12,  16,  and  14  equally-distanced  points  are  placed 
between  the  first  &  second,  second  &  third,  and  third  &  first 
anchor  points,  respectively.  Together  with  the  three  anchor 
points,  these  45  landmark  points  represent  a  sample  of  the 
shape  of  tibia. 

Let  us  use  two  vectors.  Si  and  S2,  to  describe  the 
coordinates  of  the  n  (n  =  45  in  the  current  application) 
landmark  points  of  two  shapes  in  the  training  set: 


Fig.  1  An  example  of  a  CT  image  of  a  leg, 
showing  the  detected  contour  of  tibia  and 
three  anchor  points  labeled  as  1,2,  and  3. 
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Si  =  (xu,  yn . xlk,  ylk . xln,  yln)T  (1) 

s2  =  (x2i,  y2i> . x2k,  y2k . x2n,  y2n)T  (2) 

where  (xlk,  ylk)  and  (x2k,  y2k)  are  coordinates  of  the  k’th  point 
of  shape  1  and  shape  2,  respectively.  The  difference  between 
these  two  shapes  can  be  defined  as: 

d12  =  (S2-S1)T(S2-Sl)  (3) 

We  may  consider  that  this  difference  is  contributed  by  two 
sources.  The  first  source  is  the  pose  difference  (different 
size,  orientation,  and  position)  between  the  two  shapes,  and 
the  second  source  is  the  intrinsic  shape  variation.  The  first 
source  can  be  removed  by  scaling,  rotating,  and  translating 
the  second  shape  with  respect  to  the  first  shape  in  an  attempt 
to  minimize  di2.  The  operation  of  scaling  S2  by  a  factor  s, 
rotating  by  an  angle  0,  and  translating  by  a  vector  t  can  be 
described  by  the  following  equation: 

S2*  =  M(s,  0)S2  +  t  (4) 

where  M  is  a  2x2  matrix: 


M(s,  0)  = 


scos(0) 

ssin(0) 


-ssin(0) 

scos(Q) 


(5) 


and  t  is  a  translation  vector  with  a  length  of  2n: 

1  (tx?  tys  tx5  tys . t  \  -  ty) 


(6) 


For  a  given  pair  of  Si  and  S2,  one  can  solve  for  the  pose 
parameters  s,  0,  and  t  that  minimize  d i2  [3],  When  di2  is 
minimized,  we  say  the  two  shapes  are  aligned,  and  the 
remaining  difference  indicates  the  intrinsic  shape  variation 
between  Si  and  S2. 


The  above  minimization  procedure  is  also  applied  to  S3, 
S4,  ....  and  finally  S60  so  that  the  entire  training  set  is  aligned. 


B)  Generate  a  Point  Distribution  Model  (PDM) 


The  mean  shape,  S  ,  and  the  covariance  matrix,  C,  of  the 
aligned  shapes  in  the  training  set  are  calculated  as: 

-  1  N 

S=—  5^S,-  (N  =  60  in  this  case)  (7) 
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Fig.  2  Mean  shape  of  tibia  obtained  from  60  aligned 
training  shapes,  and  45  landmarks  points.  The  1st, 

14th  and  31st  points  are  the  original  anchor  points. 

Based  on  the  principle  components  analysis,  the  modes  of 
variation  of  the  points  of  the  shape  are  represented  by  the  unit 
eigenvectors  <!>,  (i  =  1  to  2n)  of  the  covariance  matrix  C: 

C<I>,  =  V*>,  (9) 

where  A.;  is  the  i’th  ordered  eigenvalue  of  C,  A;  >Ai+i.  Since 
the  proportion  of  the  total  variance  explained  by  each 
eigenvector  is  equal  to  the  corresponding  eigenvalue,  most  of 
the  shape  variation  can  usually  be  explained  by  the  first  few 
modes.  In  our  case  of  modeling  the  shape  of  tibia,  it  is  found 
that  98%  of  the  total  shape  variation  in  our  training  set  can  be 
explained  by  the  first  6  eigenvectors.  As  a  result,  any  shape  in 
the  training  set  can  be  approximated  using  the  mean  shape 
and  a  weighted  sum  of  the  deviations  obtained  from  the  first 
6  modes: 

S  =  S  +  4>b  (10) 

where  <I>  =  ( <T>  1 ,  <I>2,...<I>6)  is  the  matrix  of  the  first  6 
eigenvectors,  and  b  =  (b|,  b2  ...b6)T  is  a  vector  of  suitable 
weights  for  individual  eigenvectors. 

Equation  (10)  allows  us  to  generate  new  examples  of  the 
shape  by  varying  the  parameters  (b,)  within  a  suitable  limit. 
Since  the  variance  of  b,  can  be  shown  to  be  A;,  a  suitable  limit 
which  is  used  in  the  current  application  is: 

-3^<b,<3^  (11) 

C)  Using  PDM  to  segment  the  targeted  structure 


C  =  i-f\S,-S)(S,.-S)r  (8) 

Ntt 

where  S;  is  the  vector  (with  a  length  of  2n)  describing  the 
coordinates  of  the  45  landmark  points  of  the  i’th  shape  in  the 
aligned  training  set  and  C  is  a  2nx2n  matrix.  Fig.  2  shows 
the  mean  shape  of  the  60  aligned  shapes  of  tibia.  The  figure 
also  shows  the  45  labeled  landmark  points.  The  three  anchor 
points  now  become  the  l’st,  14’th,  and  3 1  ’ st  landmark  points, 
respectively. 


Having  generated  PDM,  we  can  now  search  and  segment 
the  targeted  structure  in  an  image.  The  overall  approach  can 
be  summarized  as  consisting  of  two  steps.  Step  1  is  to  use 
(10)  to  generate  a  suitable  seed  model  shape  So: 

S0  =  S  +  <I>b  (12) 

Step  2  is  to  change  the  pose  of  So  so  that  the  resulted  shape  S 
best  matches  the  boundary  of  the  targeted  structure: 

S  =  M(s,  0)So  +  t  (13) 
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Since  we  already  have  S  and  <I>,  the  task  is  to  determine  the 
shape  parameter  b  and  the  pose  parameters  s,  0  and  t,  for  a 
given  image.  This  is  accomplished  by  using  the  following 
iterative  procedure. 

1)  Initialization.  Initially,  the  shape  parameter,  b,  is  set  to 
zero  in  (12).  Based  on  a  visual  inspection  of  the  targeted 
structure  (tibia)  in  the  image,  an  initial  shape  S  is  generated  to 
approximate  the  structure  by  manually  adjusting  the  pose 
parameters  s,  0,  and  t: 

S  =  M(s,  0)SO  +  t  =  M(s,  0)  S  +  t  (14) 

2)  Search  for  the  boundary  of  the  targeted  structure.  From 
each  model  point  (previously  called  the  landmark  point)  in  S, 
search  for  the  possible  edge  point  of  the  target  along  a  search 
line  that  is  perpendicular  to  the  boundary  of  S  at  that  point. 
Fig.  3  depicts  this  searching  process. 

To  determine  the  edge  point  along  the  search  line,  the  ID 
pixel  profile  is  first  smoothed  with  a  ID  Gaussian  filter.  A 
derivative  operator  is  then  used  to  compute  the  gradient 
magnitude.  Finally,  the  edge  point  is  defined  as  the  point 
where  the  gradient  magnitude  is  the  largest  and  is  also  larger 
than  a  predetermined  threshold.  This  edge  search  is 
performed  locally.  In  our  present  study,  the  search  line  has 
15  pixels  on  each  side  of  the  model  point.  If,  within  the 
search  range,  there  is  no  edge  point  that  meets  the  threshold 
requirement,  the  original  model  point  is  used  as  the  default 
edge  point. 

After  the  above  edge  searching  process  is  performed  for 
each  model  point,  a  displacement  vector  AS  =  (dx,,  dyj,  ... 
dxk,  dyk,  ...  dxn,  dyn)T  is  produced  where  (dxk,  dyk)  is  the 
difference  vector  between  the  edge  point  for  the  k’th  model 
point  and  the  k’th  model  point.  The  desired  boundary  of  the 
structure  at  this  time  is  represented  by  the  n  edge  points 
whose  coordinates  are  represented  by  vector  Y  =  (S  +  AS). 

3)  Align  the  model  shape  S  with  the  current  edge  Y.  To  align 
the  model  shape  S  with  Y,  the  pose  parameters  in  (14)  have 
to  be  further  changed  (updated)  so  that  the  difference 
between  the  new  shape  S  and  Y  is  minimized  (this  is  the 
same  problem  as  to  align  St  with  Si  discussed  earlier): 

S  =  M(s(l+ds),  (0+d0))So  +  (t+dt)  (15) 

where  ds,  d0  and  dt  are  the  additional  changes  in  the  pose 
parameters  so  that  the  difference  between  S  and  Y  is 
minimized.  However,  by  adjusting  the  pose  parameters  along 
will  not  perfectly  match  the  model  shape  with  the  edge  Y  due 
the  intrinsic  difference  between  the  original  seed  shape  S0 
and  Y.  The  remaining  difference  between  S  and  Y  can  be 
removed  only  by  adjusting  the  original  seed  shape  of  the 
model.  In  other  words,  the  following  transformation 
produces  a  model  shape  which  would  match  the  desired 
structure  Y : 

M(s(l+ds),  (0+d0))(So+dS)  +  (t+dt)  =Y  (16) 


Fig.  3  Left:  an  initial  shape  S  represented  by  45 
points,  showing  edge  searching  for  a  particular 
model  point.  Right:  possible  ID  pixel  profile 
along  the  search  line. 

It  can  be  proven  that  M_1(s,  0)  =  M(s_1,  -0).  The  required 
adjustment,  dS,  to  the  seed  shape.  So,  can  then  be  found  as: 

dS  =  M(s"1(l+ds)"1,  -(0+d0))(Y-t-dt)  -  S0  (17) 

4)  Adjust  the  shape  parameter  b.  For  the  desired  change  in 
the  seed  shape  as  defined  by  (17),  we  wish  to  find  the 
parameter  adjustment  db  so  that: 

So  +  dS  =  S  +  <I>b  +  dS  =  S  +  <I>(b+db)  (18) 

The  adjustment  db  can  then  be  solved  as: 

db  =  <I>T(dS)  =  4>VdS)  (19) 

Finally,  the  shape  parameter  is  updated  by  the  following 
equation  (with  the  constraint  defined  by  (11)): 

b  =  b  +  db  (20) 

5)  Generate  a  new  model  shape  using  the  updated  seed  model 
shape: 

S  =  M(s(l+ds),  (0+d0))(  S  +<J>(b+db))+  (t+dt)  (21) 

6)  Go  back  to  step  (2). 

The  above  procedure  is  repeated  until  the  change  in  S  at 
the  end  of  each  iteration  is  less  than  a  predetermined  value. 

III.  RESULTS 

Fig.  4  shows  an  example  of  segmenting  tibia  using  the 
ASMs  described  in  this  paper.  Fig.  4(a)  is  the  original 
ultrasound  image  of  a  healthy  volunteer’s  right  leg.  To 
produce  the  image,  an  ultrasound  transducer  rotates  360 
degrees  around  the  leg  in  a  plane  approximately  25  cm  below 
the  knee.  A  B-scan  image  is  taken  every  10  degrees.  The  36 
images  are  then  combined  to  produce  a  compound  image 
which  is  shown  in  Fig.  4(a).  In  the  image,  the  boundary  of 
tibia  is  vaguely  visible.  Fig.  4(b)  shows  an  initial  model 
shape  which  is  manually  placed  over  tibia  area.  Although  the 
model  shape  shown  in  Fig.  4(b)  appears  as  a  continuous 
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contour,  it  is  actually  represented  by  45  points,  with  the 
anchor  point  1  facing  the  fibula  bone.  Fig.  4(c)  shows  the 
final  model  shape  after  50  iterations.  It  can  be  seen  that  the 
final  model  shape  matches  the  tibia  boundary  quite  well. 


Fig.  4  An  example  of  segmenting  tibia  from 
ultrasound  image,  (a)  The  original  ultrasound 
image,  (b)  Initial  model  shape  placed  next  to 
the  expected  tibia  boundary,  (c)  Resultant 
model  shape  after  50  iterations. 


IV.  Discussion 

The  templates-based  deformable  models  seem  particularly 
suitable  for  segmenting  tissue  structures  in  medical 
ultrasound  images.  In  such  images,  the  boundaries  of  various 
organs  (such  as  ventricles,  bones,  etc.)  are  often  not  clearly 
defined  and  sometimes  are  even  incomplete.  Techniques  that 
heavily  rely  on  local  edges  may  produce  a  curve  that  strays 
far  away  from  the  true  boundary  of  the  targeted  structure. 
The  ASMs  described  in  this  paper  uses  a  training  set  to 
generate  a  mean  shape  for  tibia  that  is  allowed  to  deform 
within  a  certain  limit  derived  from  the  training  set.  As  a 
result,  the  final  result  will  unlikely  stray  far  away  from  the 
true  boundary. 

The  main  difficulty  in  using  ASMs  seems  to  be  in  the 
stage  of  preparing  the  training  set.  For  example,  it  is  difficult 
to  determine  whether  the  training  set  contains  just  enough 
samples  of  the  structure  of  interest.  If  a  particular  shape, 
which  is  not  in  the  set,  is  significantly  different  from  any 
samples  in  the  set,  then  it  will  not  be  accurately  segmented  by 
the  ASMs.  On  the  other  hand,  if  the  number  of  samples  in 
the  training  set  is  too  large,  manual  generation  of  landmark 
points  for  each  sample  becomes  very  time-consuming.  In  our 
particular  application,  the  shape  of  the  cross  section  of  tibia 
changes  significantly  from  the  proximal  end  to  the  distal  end. 
A  practical  question  then  is  whether  to  consider  the  entire 
tibia  as  one  structure  or  to  consider  the  proximal  tibia  as  one 
structure  and  the  distal  tibia  as  another  structure.  Another 
consideration  is  to  combine  the  cross  section  of  distal  tibia 
with  that  of  fibula  and  treat  the  two  contours  as  one  stmcture. 
These  are  some  topics  of  our  future  study. 
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